CLAIMS 

We claim: 

Pern for ranking items in a search result, the 
lore terms; 
; a plurality of items satisfying the query; 
in the query result, combining ratings 
users selected the item in query results 
i terms of the query and (b) levels of effort 
ombination of ratings produces a ranking 
in the ratings that are combined reflect the 
in the query results from which they were 
in the ratings that are combined reflect the 
selected items in the query results from 
in the ratings that are combined reflect the 
terns were selected were scrolled to reach 
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5. The method of claim 1 wherein the ratings that are combhied reflect, in 
the query results from which the items were selected, the number of pages of each query 
result that preceded the page of each query result containing the selected it^m. 

6. A method in a computing system for compiling statistics usable to rank 
items in a distinguished query result produced for a distinguished query, the method 
comprising: / 

receiving a rating set of queries, each query in the rating set specifying one or 
more terms; / 
for each query in the rating set, / 

generating a query result identifying one or more items satisfying the 
query; / 

allowing a user to select one or more of the items identified in the query 
result; and / 

for items selected from /the query result, for terms specified by the 
query, / 

determining an adjustment factor indicating the level of effort 
necessary to effectuate selection of the item; and 

adjusting rating score corresponding to the combination of the 
selected item and the term specified by the query by the determined adjustment factor; 

such/hat rating scores are produced that are usable to rank items 
in a distinguished query result produced for a distinguished query. 

7. The method of claim 6 wherein determining each adjustment factor 
includes increasing the Adjustment factor for items occurring in the query result before the 
selected item. / 

8. / The method of claim 6 wherein determining each adjustment factor 
includes increasing the adjustment factor for navigation operations performed to reach the 
selected item ha the query result. 



(24976-8005US03/SL01 1210.106] 



-18- 



5/2/01 



9. A computer memory containing a user behavior dat^smicture usable to 
rank the relevance of items in a query result, the data stmctur^romprising a plurality of 
rating scores, each rating score corresponding both to a^qtfery term and to an item, and 
reflecting quantitatively the extent to which usershaw selected the item from query results 
generated from queries specifying the quejyierm and the level of effort required to make 
such selections, 

such that the data strachu^Bl% be used to rank items in a distinguished query result 
produced for a distinguished query by, for each item in the distinguished query result, 
retrieving from thr'aata structure the rating scores corresponding to the item and any term 
specified in^he distinguished query and combining the retrieved rating scores to generate a 
ranking^alue for the item. 

10. A method in a computing system for assessing the usefulp^ss of a 
distinguished item relative to queries containing a distinguished query term, cpifiprising: 

from among a plurality of executed queries, identifying queries containing the 
distinguished query term in whose query results the distinguish kjem was both (a) included 
and (b) selected; / 

for each identified query, determining^ constituent score reflecting the amount 
of effort needed to select the query; and s( // 

combining the determined con^ment icore to obtain an overall score 
indicating the usefulness of the distinguished kem relative to queries containing the 
distinguished query term. / / 

11. The method/of claim 10, further comprising using the obtained overall 
score to position the distinguished item in the query result of a query containing the 
distinguished term. / 

12. / The method of claim 10 wherein the determination of each constituent 
score is based^ a position of the distinguished item within the query result of the identified 
query. / 
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13. The method of claim 10 wherein the determination of each ^Constituent 
score is based on a level of prominence of the distinguished item within the qiiery result of 
the identified query. / 

14. The method of claim 10 wherein the determination of the constituent 
score for each identified query is based on the navigation command/ issued after the query 
result is displayed and before the distinguished item is selected. / 

15. The method of claim 10 wherein the determination of the constituent 
score for each identified query is based on an amount of time that elapses between the 
display of the query result and the selection of the distinguished item. 

16. A computer-readable medium wpose contents cause a computing system 
to rate the usefulness of a distinguished item^lanve to queries containing a distinguished 
query term by: / / 

from among a pluralitWl ^ecu^ queries, identifying queries containing the 
distinguished query term in whose query results the distinguish item was both (a) included 
and (b) selected; f 

for each identified query, jaetermining a constituent score reflecting the amount 
of effort needed to select the query; atid 

combining the determined constituent score to obtain an overall score 
indicating the usefulness of the distinguished item relative to queries containing the 
distinguished query term. / 

17. A computing system for assessing the usefulness of a distinguished item 
relative to queries containing a distinguished query term, comprising: 

a query identification subsystem that identifies from among a plurality of 
executed queries queries containing the distinguished query term in whose query results the 
distinguish item Was both (a) included and (b) selected; 

a/constituent score determination subsystem that determines a constituent score 
for each identified query reflecting the amount of effort needed to select the query; and 
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an overall score determination subsystem that combines the determined constituent score to 
obtain an overall score indij^Hi^the usefulness of the distinguished item relative to queries 
containing the distmgriimedquety term. 

18. A method in a computing system ranking/items in a search result, the 
method comprising: 

receiving a query specifying one or more tends; 

generating a query result identifying a p^rality of items satisfying the query; 

and 

for a plurality of items identified id the query result, combining ratings of 
frequencies with which users selected the item m earlier queries specifying either (a) one or 
more terms of the query or (b) a term sharing a root with a term of the query to produce a 
ranking value for the item. 



19. A computer-readable medium whose contents cause a computing system 
to rank items in a search result by: 

receiving a query specifying one or more terms; 

generating a query Jesuit identifying a plurality of items satisfying the query; 

and 

for a plurality of items identified in the query result, combining ratings of 
frequencies with which usersr selected the item in earlier queries specifying either (a) one or 
more terms of the query 0/ (b) a term sharing a root with a term of the query to produce a 
ranking value for the iter 



20. Al method in a computing system for compiling statistics usable to rank 
items in a distinguished query result produced for a distinguished query, the method 
comprising: 

receiving a rating set of queries, each query in the rating set specifying one or 

more terms; 

/for each query in the rating set, 

generating a query result identifying one or more items satisfying the 

que/ ^ 
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query, adjusting a rating score corre^pcmding to the combination of the selected item and the 
root of the term specified by tiierquery, the rating score indicating the relative frequency with 
which users have seleptea the selected item when the selected item has been identified in 
search results generated from queries containing a search term sharing a root of the search 
term speci£tea by the query, to produce rating scores usable to rank items in a distinguished 
query^fesult produced for a distinguished query. 
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